As SEOs, we tend to come from an audit mindset, to look for things that are wrong and to try and decide what is the best way of doing things. In today’s episode, Will tries to get you into a testing mindset, and helps you do so by generating test ideas through ChatGPT.
Click on the whiteboard image above to open a high resolution version in a new tab!
Video Transcription
Hi, Moz fans. My name is Will Critchlow, and I’m the founder and CEO at SearchPilot, and I wanted to talk to you today about generating eight million test ideas for your website. At SearchPilot, if you don’t know us, we run SEO tests for some of the world’s biggest websites, and I think probably at this point have run as many SEO tests as anyone in the world.
The whole angle that I’m coming from here is getting ourselves into a testing mindset. So as SEOs, we have a very big temptation to come from more of an audit mindset, to look for things that are wrong and to try and decide what is the best way of doing things. I want to argue that when you’ve got testing in place, you want to move to being an explorer, not an auditor.
So you’re kind of looking for inspiration, ideas, all the different things that you can test out to see if they’re better than what you’re doing right now. We’re going to talk today just about title tags. Title tag testing is a great way to get started with SEO testing. They’re a particularly powerful form of SEO test because your title can affect what you rank for and where you rank, as well as the click-through rate that you get in the search results because it tends to show up right there in the snippet.
We’ll get on to where the eight million comes from. But I’m going to be talking a little bit about a tool that we’ll link to and you can download so that you can help yourselves do this with our tool and with our help. But we’ve put together this tool that helps you look at, say, the top 100 ranking pages for a particular query and analyze the titles of all of those pages that are ranking there.
I find this a really powerful way of doing ideation because you get to see all the things that your competitors are doing. You get to compare those and look at, “Hey, there’s an angle that I could be taking.” Or in my exploration, I can think, “Hey, that’s an interesting idea. I’m going to give that a test.” What the tool does is it basically spits out a couple of charts that look like this, where you get to look at different kinds of features of these competitors’ titles.
So it runs on a keyword, something like “hotels in London,” and you might look for features like: Does it mention London? Well, yeah, I mean, they pretty much all are going to mention London. But does it say best? Does it have reviews in there? Does it have prices in there? Does it talk about discount, all these other different features that you might see in different titles? So we spit out a bunch of charts that say relative frequency of all these different features across the top 100, across the top 20, or even one that compares what’s prevalent in the top 20 versus the bottom 20 in that 100.
It just drives off a simple spreadsheet. So it’s a simple Screaming Frog crawl. You can get started with this really quickly on your own machine. You do the crawl, crawl those titles, and pull them into an export from Screaming Frog, dump them straight into the spreadsheet. Then it’s a whole load of regular expression lookups that can say, “Does it have a pipe in there? Does it have a hyphen? Does it have a colon?
Does it have a pound sign? Does it use the word “best”? Does it talk about London and the UK and all these other features?” It pulls all this kind of stuff out, and you can customize it in the tool. But this is where the eight million comes from. Because the way I’ve put this together right now, it has 23 columns, so there are 23 different things that you may or may not include in a title, and 2 to the 23 is 8 million and a little bit.
So you even get some bonus ones above that eight million. But what I wanted to specifically talk about in this video is to get into the tactics of one specific thing that I found interestingly useful in the work of putting this together, and this is just one of the columns in the spreadsheet. So I mentioned a lot of these drive off regular expressions, just looking up a basic yes or no.
Does the title include a pound sign, for example? But there’s one particular that was slightly trickier, which is I wanted to say, “Does the title include the brand of the company?” That at first glance seems like it’s going to be easy because you’re just going from domain.com to the brand is domain, which works for something like Kayak.
Kayak.com is the domain, and you can just pull out the brand name as Kayak. But it’s not always that simple. Some of them use the whole domain name as the brand. So hotels.com, booking.com, the brand is not hotels or booking. It’s hotels.com and booking.com. Then some of them, the information isn’t even there in that domain name. So cntraveler.com for example, the brand is Condé Nast Traveler.
So there’s actually no way of doing that in a spreadsheet. You can’t just use a regular expression and pull out those bits of things because that information isn’t even there in that domain name. So this is my little tactical tip is this is a perfect place to turn to ChatGPT, because if you’re going to do this for hundreds and hundreds of domains, you don’t want to be doing this manually.
Some of them you might be able to type in and some of them you might be able to automate, and then some of them you’re just going to have to go and look up, and that is a really slow process. The industry is abuzz with ideas for using ChatGPT right now. But this is one that I found particularly interesting because it’s almost just working out of the box. It doesn’t actually need too much human oversight, unlike when you’re asking it to produce new content for you.
You can create a prompt that just works for ChatGPT. I’ve experimented with a few things, and you can start out just asking. You can say I’m going to give you a list of domains and I would like a list of brands returned for those domains, and that prompt will get you part of the way there. It works even more powerfully, I’ve found, if you give it some examples.
So just like I did here, you can give the prompt saying, for example, the brand of kayak.com is Kayak. The brand of hotels.com is hotels.com. The brand of cntraveler.com is Condé Nast Traveler. When you put all of that together, ChatGPT can straightaway out of the box give you hundreds of lookups between domain name and brand, which straightaway lets you build a spreadsheet where you don’t have to do all of that manually.
This is just, for my mind, given where artificial intelligence tools are at right now, this is one of those perfect uses because when you’ve got that big long list, it’s actually super quick to do a quick human pass over that and just look down the list and go, “Yeah, those all look right.” So much quicker than it would be to type them all out or go looking each one up individually, copying and pasting them around the place.
A really great time saver. I think it’s Benedict Evans who talks about the current state of the art in artificial intelligence is roughly it gives you 10,000 interns. Anything that you could delegate to an intern you can get ChatGPT to do. This is a perfect example. While we’re on examples of this, my other favorite one is for debugging. We talked about regular expressions, and I’m convinced that nobody really understands regular expressions.
Certainly I find, when I’m putting them together, I often make little mistakes and I need to do a little bit of debugging. I need to try and figure out why it’s not doing exactly what I wanted it to do. This is also a great use of ChatGPT. Drop your regular expression in and just ask it questions. Just say, “Why is this not picking up this example that I thought it would?” It is a great like sparring partner coming back to you with ideas and helping you figure out those things.
So that’s been a little bit of a lightning journey through how to generate eight million test ideas. If you’re into SEO testing, follow @searchpilot on Twitter and head on over to our website at searchpilot.com and sign up for the newsletter because every couple of weeks we publish a new SEO test result that helps the whole industry figure out what’s working and what’s not in Google.
So, as I said, my name is Will Critchlow. I’m the founder at SearchPilot. It’s been great talking to you today. Enjoy your day.
